Information Gain Versus Gain Ratio: A Study of Split Method Biases

One corollary of the Cullen Scha er's Conservation Law of Generalization Performance indicates that no learner is generally better than another learner. If the rst learner performs better than the second learner on some learning situations, the rst learner must perform worse than the second learner on other learning situations. Unfortunately, the corollary does not provide a description of the circumstances where a speci c learner has an advantage. This article focuses on two decision tree learners. One uses the information gain split method and the other uses gain ratio. It presents a predictive method that helps to characterize problems where information gain performs c 2001 The MITRE Corporation. All Rights Reserved. better than gain ratio (and vice versa). To support the practical relevance of this research, it shows that the predictive method works e ectively on the contraceptive method choice problem from the Cal-Irvine Machine Learning Repository. This article brings new insight on how these two split methods a ect a decision tree learner's bias.